Sustavi nelinearnih jednadžbi

Problem. Nađimo rješenje $ξ = (ξ_{1}, ξ_{2}, \dots, ξ_{n})$ sustava od $n$ jednadžbi

$\begin{aligned} f_{1} (x) & = 0, \\ f_{2} (x) & = 0, \\ ⋮ \\ f_{n} (x) & = 0, \end{aligned}$

i $n$ nepoznanica $x = (x_{1}, x_{2}, \dots, x_{n})$ . Uz oznaku $f = (f_{1}, f_{2}, \dots, f_{n})^{T}$ , ovaj sustav možemo zapisati kao

$f (x) = 0.$

Opisat ćemo Newtonovu metodu i tri kvazi-Newtonove metode:

Broydenovu metodu,
Davidon-Fletcher-Powell metodu i
Broyden-Fletcher-Goldfarb-Schano metodu.

Sve metode, uz zadanu početnu aproksimaciju $x^{(0)}$ , generiraju niz točaka $x^{(n)}$ koji, uz određene uvjete, konvergira prema rješenju $ξ$ .

Napomena. Opisi metoda i primjeri se nalaze u knjizi Numerička matematika, poglavlje 4.4.

24.7 μs

Newtonova metoda

Jacobijan ili Jacobijeva matrica funkcija $f$ u točki $x$ je matrica prvih parcijalnih derivacija

$J (f, x) = (\begin{matrix} \frac{\partial f_{1} (x)}{\partial x_{1}} & \frac{\partial f_{1} (x)}{\partial x_{2}} & \dots & \frac{\partial f_{1} (x)}{\partial x_{n}} \\ \frac{\partial f_{2} (x)}{\partial x_{1}} & \frac{\partial f_{2} (x)}{\partial x_{2}} & \dots & \frac{\partial f_{2} (x)}{\partial x_{n}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{\partial f_{n} (x)}{\partial x_{1}} & \frac{\partial f_{n} (x)}{\partial x_{2}} & \dots & \frac{\partial f_{n} (x)}{\partial x_{n}} \end{matrix}) .$

Za zadanu početnu aproksimaciju $x^{(0)}$ , računamo niz točaka

$x^{(k + 1)} = x^{(k)} - s^{(k)}, k = 0, 1, 2, \dots,$

gdje je $s^{(k)}$ rješenje sustava

$J (f, x^{(k)}) \cdot s = f (x^{(k)}) .$

Za računanje Jacobijana koristimo paket ForwardDiff.jl. Za crtanje funkcija koristimo paket Plots.jl.

12.0 μs

xxxxxxxxxx
 
begin
    using ForwardDiff
    using Plots
    plotly()
    using LinearAlgebra
end

22.9 s

Newton (generic function with 2 methods)

xxxxxxxxxx
 
function Newton(f::Function,J::Function,x::Vector{T},ϵ::Float64=1e-10) where T
    iter=0
    s=ones(T,size(x))
    ξ=x
    while norm(s)>ϵ && iter<100
        s=J(x)\f(x)
        ξ=x-s
        iter+=1
        x=ξ
    end
    ξ,iter
end

78.6 μs

Primjer 1

(Dennis i Schnabel, 1996) Rješenja sustava

$\begin{aligned} 2 (x + y)^{2} + (x - y)^{2} - 8 & = 0 \\ 5 x^{2} + (y - 3)^{2} - 9 & = 0 \end{aligned}$

su točke $T_{1} = (1, 1)$ i $T_{2} \approx (- 1.18, 1.59)$ .

6.1 μs

f₁ (generic function with 1 method)

xxxxxxxxxx
 
# Vektorska funkcija
f₁(x)=[2(x[1]+x[2])^2+(x[1]-x[2])^2-8,5*x[1]^2+(x[2]-3)^2-9]

83.3 μs

Float641

11.0

-3.0

xxxxxxxxxx
 
f₁((1.0,2))

500 ns

Nacrtajmo funkcije i konture kako bi mogli približno locirati nul-točke:

3.2 μs

xxxxxxxxxx
 
begin
    # Broj točaka
    m=101
    X=range(-2,stop=3,length=m)
    Y=range(-2,stop=3,length=m)
    # Prva aplikata
    surface(X,Y,(x,y)->f₁([x,y])[1],xlabel="x",ylabel="y",colorbar=false)
    # Druga aplikata
    surface!(X,Y,(x,y)->f₁([x,y])[2],seriescolor=:blues)
end

7.8 s

xxxxxxxxxx
 
begin
    # Odredimo rješenja pomoću kontura
    contour(X,Y,(x,y)->f₁([x,y])[1],contour_labels=true)
    contour!(X,Y,(x,y)->f₁([x,y])[2],contour_labels=true)
end

406 ms

xxxxxxxxxx
 
# Jasnija slika
contour!(clims=(0,0.01),xlabel="x",ylabel="y",colorbar=:none)

656 ms

Vidimo da su nul-točke približno $T_{1} = (- 1.2, 1.5)$ i $T_{2} = (1, 1)$ . Štoviše, $T_{2}$ je točno jednaka $(1, 1)$ (1 iteracija u trećem primjeru). Nadalje, metoda ne mora konvergirati (četvrti primjer).

3.9 μs

J₁ (generic function with 1 method)

xxxxxxxxxx
 
J₁(x)=ForwardDiff.jacobian(f₁,x)

25.5 μs

2×2 Array{Float64,2}:
 10.0  14.0
 10.0  -2.0

xxxxxxxxxx
 
# Na primjer
J₁([1.0,2])

1.2 s

11Float641

-1.18347

1.58684

21Float641

1.0

1.0

31Float641

1.0

1.0

41Float641

NaN

NaN

xxxxxxxxxx
 
Newton(f₁,J₁,[-1.0,0.0]), Newton(f₁,J₁,[0.5,1.1]), 
Newton(f₁,J₁,[1.0,1.0]), Newton(f₁,J₁,[0.0,0.0])

5.1 s

Primjer 2

(Dennis i Schnabel, 1996) Rješenja sustava

$\begin{aligned} x_{1}^{2} - x_{2}^{2} - 2 & = 0 \\ e^{x_{1} - 1} + x_{2}^{3} - 2 & = 0 \end{aligned}$

su točke $T_{1} = (1, 1)$ i $T_{2} \approx (- 0.71, 1.22)$ .

6.3 μs

xxxxxxxxxx
 
begin
    f₂(x)=[x[1]^2+x[2]^2-2,exp(x[1]-1)+x[2]^3-2]
    contour(X,Y,(x,y)->f₂([x,y])[1],contour_labels=true)
    contour!(X,Y,(x,y)->f₂([x,y])[2],contour_labels=true)
    contour!(clims=(0,0.01),xlabel="x",ylabel="y",colorbar=:none)
end

246 ms

11Float641

-0.713747

1.22089

21Float641

1.0

1.0

xxxxxxxxxx
 
begin
    J₂(x)=ForwardDiff.jacobian(f₂,x)
    Newton(f₂,J₂,[-1.0,1]), Newton(f₂,J₂,[0.8,1.2])
end

17.4 ms

Primjer 3

(Dennis i Schnabel, 1996) Zadan je problem $f (x) = 0$ , gdje je

$f (x) = (\begin{matrix} x_{1} \\ x_{2}^{2} + x_{2} \\ e^{x_{3}} - 1 \end{matrix}) .$

Točna rješenja su $T_{1} = (0, 0, 0)$ i $T_{2} = (0, - 1, 0)$ . Izračunat ćemo nul-točke s nekoliko početnih aproksimacija.

26.4 μs

J₃ (generic function with 1 method)

xxxxxxxxxx
 
begin
    f₃(x)=[x[1],x[2]^2+x[2],exp(x[3])-1]
    J₃(x)=ForwardDiff.jacobian(f₃,x)
end

85.6 μs

11Float641

0.0

0.0

0.0

21Float641

0.0

0.0

7.78375e-17

31Float641

0.0

0.0666667

NaN

41Float641

0.0

-1.0

0.0

xxxxxxxxxx
 
Newton(f₃,J₃,[-1.0,1.0,0.0]),Newton(f₃,J₃,[1.0,1,1]),
Newton(f₃,J₃,[-1.0,1,-10]),Newton(f₃,J₃,[0.5,-1.5,0])

763 ms

Primjer 4

(Rosenbrock parabolic valley) Zadana je funkcija

$f (x) = 100 (x_{2} - x_{1})^{2} + (1 - x_{1})^{2} .$

Tražimo moguće ekstreme funkcije, odnosno želimo riješiti jednadžbu

$g r a d f (x) = 0.$

30.9 μs

f₄ (generic function with 1 method)

xxxxxxxxxx
 
f₄(x)=100(x[2]-x[1]^2)^2+(1-x[1])^2

30.6 μs

xxxxxxxxxx
 
# Nacrtajmo funkciju koristeći X i Y iz Primjera 1
surface(X,Y,(x,y)->f₄([x,y]), seriescolor=:blues, xlabel="x", ylabel="y")

222 ms

xxxxxxxxxx
 
begin
    # Funkcija je zahtjevna u smislu određivanje ekstrema
    g₄(x)=ForwardDiff.gradient(f₄,collect(x))
    contour(X,Y,(x,y)->g₄([x,y])[1],contour_labels=true)
    contour!(X,Y,(x,y)->g₄([x,y])[2],contour_labels=true)
    contour!(clims=(-0.5,0.5),xlabel="x",ylabel="y",colorbar=:none)
end

1.1 s

Iz kontura vidimo da je primjer numerički zahtjevan, dok analitički lako vidimo da je jedina nul-točka $x_{1} = (1, 1)$ .

U ovom primjeru je vektorska funkcija $f$ zadana kao gradijent skalarne funkcije pa Jacobijevu matricu računamo korištenjem funkcije FowardDiff.hessian() koja računa aproksimaciju matrice drugih parcijalnih derivacija polazne funkcije $f$ .

6.9 μs

1Float641

1.0

1.0

xxxxxxxxxx
 
Newton(g₄,x->ForwardDiff.hessian(f₄,x),[-1.0,2.0])

1.1 s

Primjer 5

Zadana je fukcija

$f (x) = \sum_{i = 1}^{11} (x_{3} \cdot \exp (- \frac{(t_{i} - x_{1})^{2}}{x_{2}}) - y_{i})^{2},$ gdje su brojevi $(t_{i}, y_{i})$ zadani tablicom: $\begin{array}{cccccccccccc} i & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 \\ t_{i} & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 \\ y_{i} & 0.001 & .01 & .04 & .12 & .21 & .25 & .21 & .12 & .04 & .01 & .001 \end{array}$

Želimo riješiti jednadžbu

$g r a d f (x) = 0.$ Za razliku od prethodnih zadataka, gdje je kondicija $κ (J) = O (10)$

u zadacima (a), (b) i (c) i

$κ (J) = O (1000)$

u zadatku (d), u ovom zadatku je

$κ (J) > O (10^{6})$

pa je metoda netočna i ne konvergira prema točnom rješenju $x = (4.93, 2.62, 0.28)$ .

7.5 μs

f₅ (generic function with 1 method)

xxxxxxxxxx
 
begin
    t=collect(0:10)
    y=[0.001,0.01,0.04,0.12,0.21,0.25,0.21,0.12,0.04,0.01,0.001]
    f₅(x)=sum([( x[3]*exp(-((t[i]-x[1])^2/x[2]))-y[1])^2 for i=1:11])
end

2.9 ms

0.157753

2Float641

2.71553e-6

0.029986

1.13247

1.73704e5

xxxxxxxxxx
 
begin
    # Početna točka je vrlo blizu rješenja
    x₀=[4.9,2.63,0.28]
    f₅(x₀)
    g₅(x)=ForwardDiff.gradient(f₅,x)
    J₅(x)=ForwardDiff.hessian(f₅,x)
    f₅(x₀), g₅(x₀), cond(J₅(x₀))
end

2.3 s

1Float641

6.50244

0.00245085

-1.60888e-7

xxxxxxxxxx
 
x₅,iter₅=Newton(g₅,J₅,x₀,1e-8)

18.4 ms

Float641

1.52356e-51

2.04301e-49

-3.07371e-47

xxxxxxxxxx
 
g₅(x₅)

42.0 μs

1Float641

NaN

NaN

NaN

xxxxxxxxxx
 
Newton(g₅,J₅,[4.9,2.62,0.28],1e-8)

508 μs

Broydenova metoda

Za odabranu početnu aproksimaciju $x_{0}$ i matricu $B_{0}$ , za $k = 0, 1, 2, \dots$ , računamo redom:

$\begin{aligned} B_{k} \cdot s_{k} & = - f (x_{k}) (sustav) \\ x_{k + 1} & = x_{k} + s_{k} \\ y_{k} & = f (x_{k + 1}) - f (x_{k}) \\ B_{k + 1} & = B_{k} + \frac{(y_{k} - B_{k} s_{k}) s_{k}^{T}}{s_{k} \cdot s_{k}} \end{aligned}$

Na ovaj način izbjegavamo računanje Jacobijeve matrice u svakom koraku. Možemo uzeti $B_{0} = J (x_{0})$ , ali i neku drugu matricu.

23.4 μs

Broyden (generic function with 2 methods)

xxxxxxxxxx
 
function Broyden(f::Function,B::Matrix,x::Vector{T},ϵ::Float64=1e-10) where T
    iter=0
    s=ones(T,length(x))
    ξ=x
    while norm(s)>ϵ && iter<100
        s=-(B\f(x))
        ξ=x+s
        y=f(ξ)-f(x)
        B=B+(y-B*s)*(s/(s⋅s))'
        x=ξ
        iter+=1
    end
    ξ,iter
end

58.3 μs

11Float641

-1.18347

1.58684

21Float641

1.0

1.0

xxxxxxxxxx
 
# Primjer 1
Broyden(f₁,J₁([-1.0,0.0]),[-1.0,0.0]), 
Broyden(f₁,J₁([1.0,1.5]),[1.0,1.5])

775 ms

11Float641

-0.0356391

-2.53166

21Float641

0.916022

-3.04547

31Float641

-1.18347

1.58684

xxxxxxxxxx
 
begin
    # Objasnite ponašanje metode kada za početnu matricu uzmemo jediničnu matricu!
    eye(n)=Matrix{Float64}(I,n,n)
    Broyden(f₁,eye(2),[-1.0,0.0]), Broyden(f₁,eye(2),[1.0,1.5]),
    Broyden(f₁,eye(2),[-1,1.5])
end

31.5 ms

11Float641

-0.713747

1.22089

21Float641

1.0

1.0

xxxxxxxxxx
 
begin
    # Primjer 2
    x0=[-1.0,1]
    x1=[0.8,1.2]
    Broyden(f₂,J₂([-1.0,1]),[-1.0,1]), Broyden(f₂,J₂([0.8,1.2]),[0.8,1.2])
end

85.7 μs

11Float641

0.0

5.96536e-26

0.0

21Float641

0.0

-1.0

0.0

xxxxxxxxxx
 
# Primjer 3
Broyden(f₃,J₃([-1.0,1,0]),[-1.0,1,0]), Broyden(f₃,J₃([0.5,-1.5,0]),[0.5,-1.5,0])

16.6 ms

11Float641

0.834064

0.695042

21Float641

1.0

1.0

31Float641

1.0

1.0

xxxxxxxxxx
 
# Primjer 4
Broyden(g₄,(x->ForwardDiff.hessian(f₄,x))([-1.0,2]),[-1.0,2]), # ali
Broyden(g₄,(x->ForwardDiff.hessian(f₄,x))([1,2.0]),[-1.0,2]),
Broyden(g₄,(x->ForwardDiff.hessian(f₄,x))([0.8,0.5]),[0.8,0.5])

40.4 ms

1Float641

18.9003

1.32399

0.0665517

xxxxxxxxxx
 
begin
    # Primjer 5
    x₆,iter₆=Broyden(g₅,(x->ForwardDiff.hessian(f₅,x))([4.9,2.6,0.2]), [4.9,2.6,0.2])
end

18.4 ms

Float641

1.85527e-29

-6.2359e-29

-2.07346e-29

xxxxxxxxxx
 
g₅(x₆)

34.9 μs

Davidon-Fletcher-Powell (DFP) metoda

DFP je optimizacijska metoda koja traži točke ekstrema funkcije $F : R^{n} \to R$ , u kojem slučaju je $f (x) = g r a d F (x)$ .

Za odabranu početnu aproksimaciju $x_{0}$ i matricu $H_{0}$ , za $k = 0, 1, 2, \dots$ , računamo redom:

$\begin{aligned} s_{k} & = - H_{k} f (x_{k}) \\ β_{k} & = \underset{β}{a r g m i n} F (x_{k} + β s_{k}) \\ s_{k} & = β_{k} s_{k} \\ x_{k + 1} & = x_{k} + s_{k} \\ y_{k} & = f (x_{k + 1}) - f (x_{k}) \\ H_{k + 1} & = H_{k} + \frac{s_{k} s_{k}^{T}}{y_{k} \cdot s_{k}} - \frac{H_{k} y_{k} y_{k}^{T} H_{k}}{y_{k} \cdot (H_{k} y_{k})} . \end{aligned}$

Za matricu $H_{0}$ možemo uzeti jediničnu matricu, a za izvršavanje iteracije nije potrebno rješavati sustav linearnih jednadžbi, već se sva ažuriranja vrše s $O (n^{2})$ operacija.

Jednodimenzionalnu minimizaciju po pravcu $x_{k} + β s_{k}$ računamo tako što metodom bisekcije nađemo nul-točke usmjerene derivacije.

11.4 μs

Bisekcija (generic function with 2 methods)

xxxxxxxxxx
 
function Bisekcija(f::Function,a::T,b::T,ϵ::Float64=1e-10) where T
    fa=f(a)
    fb=f(b)
    x=T
    fx=T
    if fa*fb>zero(T)
        # return "Netočan interval"
        if abs(fa)>abs(fb)
            return b,fb,0
        else
            return a,fa,0
        end
    end
    iter=0
    while b-a>ϵ && iter<1000
        x=(b+a)/2.0
        fx=f(x)
        if fa*fx<zero(T)
            b=x
            fb=fx
        else
            a=x
            fa=fx
        end
        iter+=1
        # @show x,fx
    end
    x,fx,iter
end

54.5 μs

DFP (generic function with 2 methods)

xxxxxxxxxx
 
function DFP(f::Function,H::Matrix,x::Vector{T},ϵ::Float64=1e-10) where T
    iter=0
    s=ones(T,length(x))
    ξ=x
    while norm(s)>ϵ && iter<50
        s=-H*f(x)
        s0=s/norm(s)
        F(ζ)=f(x+ζ*s)⋅s0
        β,fx,iterb=Bisekcija(F,0.0,1.0,10*eps())
        s*=β
        ξ=x+s
        y=f(ξ)-f(x)
        z=H*y
        H=H+(s/(y⋅s))*s'-(z/(y⋅z))*z'
        x=ξ
        iter+=1
    end
    ξ,iter
end

102 μs

Primjer. Nađimo točku ekstrema funkcije

$f (x, y) = (x + 2 y - 7)^{2} + (2 x + y - 5)^{2} .$

Funkcija ima minimum u točki $(1, 3)$ .

7.3 μs

f₆ (generic function with 1 method)

xxxxxxxxxx
 
f₆(x) = (x[1] + 2*x[2]-7)^2 + (2*x[1] + x[2]-5)^2

56.4 μs

xxxxxxxxxx
 
f₆([1,2])

1.3 μs

g₆ (generic function with 1 method)

xxxxxxxxxx
 
g₆(x)=ForwardDiff.gradient(f₆,x)

18.4 μs

1Float641

1.0

3.0

xxxxxxxxxx
 
DFP(g₆,eye(2),[0.8,2.7],eps())

658 ms

1Float641

1.0

1.0

xxxxxxxxxx
 
# Primjer 4
DFP(g₄,eye(2),[0.9,1.1])

58.8 ms

1Float641

4.89611

46.1979

0.00158913

xxxxxxxxxx
 
# Primjer 5
DFP(g₅,eye(3),[4.9,2.6,0.2])

50.4 ms

Broyden-Fletcher-Goldfarb-Schano (BFGS) metoda

BFGS je optimizacijska metoda koja uspješno traži točke ekstrema funkcije $F : R^{n} \to R$ , u kojem slučaju je $f (x) = g r a d F (x)$ .

Metoda je slična DFP metodi, s nešto boljim svojstvima konvergencije.

Neka je zadana funkcija $F : R^{n} \to R$ , čiji minimum tražimo, i neka je $f (x) = g r a d F (x)$ .

Za odabranu početnu aproksimaciju $x_{0}$ i matricu $H_{0}$ , za $k = 0, 1, 2, \dots$ , računamo redom:

$\begin{aligned} s_{k} & = - H_{k} f (x_{k}) \\ β_{k} & = a r g m i n F (x_{k} + β_{k} s_{k}) \\ s_{k} & = β_{k} s_{k} \\ x_{k + 1} & = x_{k} + s_{k} \\ y_{k} & = f (x_{k + 1}) - f (x_{k}) \\ H_{k + 1} & = (I - \frac{s_{k} y_{k}^{T}}{y_{k} \cdot s_{k}}) H_{k} (I - \frac{y_{k} s_{k}^{T}}{y_{k} \cdot s_{k}}) + \frac{s_{k} s_{k}^{T}}{y_{k} \cdot s_{k}} . \end{aligned}$

Za matricu $H_{0}$ možemo uzeti jediničnu matricu, a za izvršavanje iteracije nije potrebno rješavati sustav linearnih jednadžbi, već se sva ažuriranja vrše s $O (n^{2})$ operacija.

Jednodimenzionalnu minimizaciju po pravcu $x_{k} + β s_{k}$ računamo tako što metodom bisekcije tražimo nul-točke usmjerene derivacije.

11.1 μs

BFGS (generic function with 2 methods)

xxxxxxxxxx
 
function BFGS(f::Function,H::Matrix,x::Vector{T},ϵ::Float64=1e-10) where T
    iter=0
    s=ones(T,length(x))
    ξ=x
    while norm(s)>ϵ && iter<50
        s=-H*f(x)
        s0=s/norm(s)
        F(ζ)=f(x+ζ*s)⋅s0
        β,fx,iterb=Bisekcija(F,0.0,1.0,10*eps())
        s*=β
        ξ=x+s
        y=f(ξ)-f(x)
        z=H*y
        α=y⋅s
        s1=s/α
        H=H-s1*z'-z*s1'+s1*(y⋅z)*s1'+s1*s'
        x=ξ
        iter+=1
    end
    ξ,iter
end

115 μs

1Float641

1.0

3.0

xxxxxxxxxx
 
BFGS(g₆,eye(2),[0.8,2.7],eps())

282 ms

1Float641

1.0

1.0

xxxxxxxxxx
 
# Primjer 4
BFGS(g₄,eye(2),[0.9,1.1])

48.8 ms

1Float641

2.08908

31226.6

0.00100056

xxxxxxxxxx
 
# Primjer 5
BFGS(g₅,eye(3),[4.9,2.6,0.2])

54.6 ms

Julia paketi

Prethodni programi su jednostavne ilustrativne implementacije navedenih algoritama. Julia ima paket NLsolve.jl za rješavanje sustava nelineranih jednadžbi i paket Optim.jl za nelinearnu optimizaciju.

7.3 μs

xxxxxxxxxx
 
using NLsolve

21.4 s

f₁! (generic function with 1 method)

xxxxxxxxxx
 
# Primjer 1
function f₁!(fvec,x)
    fvec[1] = 2(x[1]+x[2])^2+(x[1]-x[2])^2-8
    fvec[2] = 5*x[1]^2+(x[2]-3)^2-9
end

30.8 μs

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [-1.0, 0.0]
 * Zero: [-1.1834670032425283, 1.5868371427230779]
 * Inf-norm of residuals: 0.000000
 * Iterations: 5
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 6
 * Jacobian Calls (df/dx): 6

xxxxxxxxxx
 
nlsolve(f₁!,[-1.0,0])

552 ms

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [0.5, 1.1]
 * Zero: [1.0000000000000002, 1.0000000000000002]
 * Inf-norm of residuals: 0.000000
 * Iterations: 5
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 6
 * Jacobian Calls (df/dx): 6

xxxxxxxxxx
 
nlsolve(f₁!,[0.5,1.1])

44.2 μs

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [-1.0, 1.0]
 * Zero: [-0.7137474114758742, 1.2208868222037403]
 * Inf-norm of residuals: 0.000000
 * Iterations: 4
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 5
 * Jacobian Calls (df/dx): 5

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [0.8, 1.2]
 * Zero: [0.9999999999940328, 1.0000000000115203]
 * Inf-norm of residuals: 0.000000
 * Iterations: 4
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 5
 * Jacobian Calls (df/dx): 5

xxxxxxxxxx
 
begin
    # Primjer 2
    function f₂!(fvec,x)
        fvec[1] = x[1]^2+x[2]^2-2
        fvec[2] = exp(x[1]-1)+x[2]^3-2
    end
    nlsolve(f₂!,[-1.0,1]), nlsolve(f₂!,[0.8,1.2])
end

146 ms

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [-1.0, 1.0, 0.0]
 * Zero: [0.0, 2.3283064364709337e-10, 0.0]
 * Inf-norm of residuals: 0.000000
 * Iterations: 5
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 6
 * Jacobian Calls (df/dx): 6

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [1.0, 1.0, 1.0]
 * Zero: [0.0, 2.3283064364709337e-10, 1.223235687436471e-12]
 * Inf-norm of residuals: 0.000000
 * Iterations: 5
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 6
 * Jacobian Calls (df/dx): 6

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [-1.0, 1.0, -10.0]
 * Zero: [0.0, 4.9978109751571114e-11, 8.131707812509303e-17]
 * Inf-norm of residuals: 0.000000
 * Iterations: 20
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 21
 * Jacobian Calls (df/dx): 8

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [0.5, -1.5, 0.0]
 * Zero: [0.0, -1.0000000000000007, 0.0]
 * Inf-norm of residuals: 0.000000
 * Iterations: 5
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 6
 * Jacobian Calls (df/dx): 6

xxxxxxxxxx
 
begin
    # Primjer 3
    function f₃!(fvec,x)
        fvec[1] = x[1]
        fvec[2] = x[2]^2+x[2]
        fvec[3] = exp(x[3])-1
    end
    nlsolve(f₃!,[-1.0,1.0,0.0]), nlsolve(f₃!,[1.0,1,1]),
    nlsolve(f₃!,[-1.0,1,-10]), nlsolve(f₃!,[0.5,-1.5,0])
end

184 ms

xxxxxxxxxx
 
using Optim

10.0 s

 * Status: success

 * Candidate solution
    Final objective value:     5.375030e-17

 * Found with
    Algorithm:     BFGS

 * Convergence measures
    |x - x'|               = 5.13e-09 ≰ 0.0e+00
    |x - x'|/|x'|          = 5.13e-09 ≰ 0.0e+00
    |f(x) - f(x')|         = 9.67e-17 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 1.80e+00 ≰ 0.0e+00
    |g(x)|                 = 2.10e-11 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    35
    f(x) calls:    102
    ∇f(x) calls:   102

xxxxxxxxxx
 
# Primjer 4
optimize(f₄,[-1.0,2],Optim.BFGS())

1.2 s

 * Status: success

 * Candidate solution
    Final objective value:     3.572548e-12

 * Found with
    Algorithm:     BFGS

 * Convergence measures
    |x - x'|               = 1.87e+05 ≰ 0.0e+00
    |x - x'|/|x'|          = 8.70e-02 ≰ 0.0e+00
    |f(x) - f(x')|         = 6.03e-16 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 1.69e-04 ≰ 0.0e+00
    |g(x)|                 = 9.69e-09 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    41
    f(x) calls:    137
    ∇f(x) calls:   137

xxxxxxxxxx
 
# Primjer 5 - opet ne konvergira prema rješenju
optimize(f₅,[4.9,2.6,0.2],Optim.BFGS())

87.0 ms

Enter cell code...xxxxxxxxxx
 
​

200 ns